Quantile-based distributional RL agents are usually improved by using alternate L2-based losses, at the cost of theoretical validity and distributional collapse. We propose a way to leverage effective L2 losses while maintaing an estimation of the full distribution of returns.
We built a simulator that accounts for the dynamic and interactive nature of recommender systems. Off-the-shelf RL baselines are not able to solve the tasks we propose.
We discussed offline evaluation for RL-based recommender systems and noted that the most common evaluation protocol, i.e., next-item prediction, is unsuited to such approaches.
Learning to recommend slates with RL is hard, because of the combinatorial action-space. Instead of making assumptions on user behavior, we tried taking actions in the latent space of a VAE, which then decodes slates.